striatum encodes net expected return , critical for energizing performance

Decision making requires an actor to not only steer behavior towards specific goals, but also determine the optimal vigor of performance. Current research and models have largely focused on the former problem of how actions are directed, while overlooking the latter problem of how they are energized. Here, we designed a self-paced decision-making paradigm that showed that rats' performance vigor globally fluctuates with the net value of their options, suggesting that they maintain long-term estimates of the value of their current state. Lesions of the dorsomedial (DMS), and to a lesser degree, in the ventral striatum (VS) impaired such state-dependent modulation of vigor, rendering vigor to depend more exclusively on the outcomes of immediately preceding trials. The lesions, however, spared choice biases. Neuronal recordings showed that the DMS is enriched with net-value-coding neurons. In sum, the DMS encodes one's net expected return, which drives the general motivation to perform. Given stock options A and B, which one do you choose? The field of decision making and neuroeconomics have made significant progress on this problem of `action selection'1,2. Optimal behavioral selection, however, not only depends on the ability to choose which action to perform but also the appropriate vigor to perform. For example, it may be wise to flexibly adjust one's motivation to invest according to the overall state of the stock market, such as investing with lower frequency during an economic crisis. The importance of properly regulating response vigor becomes apparent when one considers costs associated with performing an action. Rapid responding may increase the rate of obtaining rewards but may also increase energetic costs. Conversely, slow responding may be energetically efficient, yet it delays all future rewards. Importantly, the cost of delaying future rewards critically depends on the net expected future reward given the current state of the animal. It is thus proposed that selection of response vigor should depend on the average or net expected reward (or “state value”) while action selection depends on values specific to individual options (and the relative value between them)3–5. This idea echoes two aspects of motivation proposed in classic animal psychology: the motivation to steer towards making a specific action (the “directing” effect or action-specific motivation) and the Correspondence should be addressed to N.U. ([email protected]).. Supplementary information is available in the online version of the paper. AUTHOR CONTRIBUTIONS A.W. wrote the manuscript, performed the experiments, performed the analyses, and was involved in the experimental design. K.M. performed the simulations. N.U. was involved in preparing the manuscript and experimental design. COMPETING FINANCIAL INTERESTS The authors declare no competing financial interests. NIH Public Access Author Manuscript Nat Neurosci. Author manuscript; available in PMC 2013 November 01. Published in final edited form as: Nat Neurosci. 2013 May ; 16(5): . doi:10.1038/nn.3377. N IH PA Athor M anscript N IH PA Athor M anscript N IH PA Athor M anscript motivation to generally “arouse” or speed-up all pre-potent actions in a non-specific manner (the “energizing” effect or action-general motivation)3,6,7. It should be noted that the directing effect may also speed-up actions towards particular goals but the energizing effect acts diffusely on a wider set of actions. Experimentally, many classic studies in animal psychology have shown that response vigor is modulated by the rate of reward, providing some limited support for the energizing effects of average reward rate8. More recently, it has been shown that manipulations of the size or probability of rewards affect choice direction and latencies in various choice tasks, which highlights motivation's directing effects9–11. However, whether response vigor is indeed regulated by average or net expected reward, that is, whether motivation energizes behavior in a global manner, remains controversial. It is believed that the basal ganglia play important roles in action selection1,2,12,13. Some studies on patients with Parkinson's Disease and lesion studies using animal models, however, have suggested that the basal ganglia also play a prominent role in the regulation of response vigor14. Mounting evidence suggests that specific areas of the striatum encode specific types of values and regulate distinct aspects of value-dependent behavior15–17. Historically, the striatum, particularly the VS has been linked to motivation18,19, although other studies implicate the role of dorsal striatum in motivation20–22. However, previous studies have not separated the directing versus energizing aspects of motivation, therefore whether these processes can be mapped onto specific parts of the striatum remains unknown. To address these questions, we designed a task that allows us to study both the directing and energizing aspects of behavioral regulation. We first examined whether response vigor is indeed modulated by net expected future rewards and how this process is separable from the directing effects that are specific to individual actions. Second, using lesions, we examined which part of the striatum is involved in the regulation of net value-dependent response vigor. Finally, we recorded the activity of single neurons in DMS and VS. The results demonstrate a critical role of the DMS in net-value dependent regulation of response vigor.